ffiEE TRANSACTIONS ON INFORMATION THEORY, VOL. XX, NO. XX, XXXX 2009 



Multiple-Description Coding by Dithered 
Delta-Sigma Quantization 



Jan 0stergaard and Ram Zamir 



Abstract 

We address the connection between the muhiple-description (MD) problem and Delta-Sigma quanti- 
zation. The inherent redundancy due to oversampling in Delta-Sigma quantization, and the simple linear- 
additive noise model resulting from dithered lattice quantization, allow us to construct a symmetric and 
time-invariant MD coding scheme. We show that the use of a noise shaping filter makes it possible 
to trade off central distortion for side distortion. Asymptotically as the dimension of the lattice vector 
quantizer and order of the noise shaping filter approach infinity, the entropy rate of the dithered Delta- 
Sigma quantization scheme approaches the symmetric two-channel MD rate-distortion function for a 
memoryless Gaussian source and MSE fidelity criterion, at any side-to-central distortion ratio and any 
resolution. In the optimal scheme, the infinite-order noise shaping filter must be minimum phase and 
have a piece-wise flat power spectrum with a single jump discontinuity. An important advantage of the 
proposed design is that it is symmetric in rate and distortion by construction, so the coding rates of the 
descriptions are identical and there is therefore no need for source splitting. 

Index Terms 

delta-sigma modulation, dithered lattice quantization, entropy coding, joint source-channel coding, 
multiple-description coding, vector quantization. 



I. Introduction and Motivation 

Delta-Sigma analogue to digital (A/D) conversion is a technique where the input signal is highly over- 
sampled before being quantized by a low resolution quantizer. The quantization noise is then processed 
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by a noise shaping filter which reduces the energy of the so-called in-band noise spectrum, i.e. the part of 
the noise spectrum which overlaps the spectrum of the input signal. The end result is high bit-accuracy 
(A/D) conversion even in the presence of imperfections in the analogue components of the system, of. [1]. 

The process of oversampling and use of feedback to reduce quantization noise is not limited to A/D 
conversion of continuous-time signals but is in fact equally apphcable to, for example, discrete time 
signals in which case we will use the term Delta-Sigma quantization. Hence, given a discrete time signal 
we can apply Delta-Sigma quantization in order to discretize the amplitude of the signal and thereby 
obtain a digital signal. It should be clear that the process of oversamphng is not required in order to obtain 
a digital signal. However, oversampling leads to a controlled amount of redundancy in the digital signal. 
This redundancy can be exploited in order to achieve a certain degree of robustness against inaccuracy in 
the quantization, or partial loss of information due to transmission of the digital signal over error-prone 
channels. In this paper we pursue the latter aspect, and relate it to the problem of multiple descriptions. 

In the information theory community the problem of quantization is usually referred to as a source 
coding problem whereas the problem of reliable transmission is referred to as a channel coding problem. 
Their combination then forms a joint source-channel coding problem. The multiple-description (MD) 
problem [2], which has recently received a lot of attention, is basically a joint source-channel coding 
problem. The MD problem is concerned with lossy encoding of information for transmission over an 
unreliable K-channel communication system. The channels may break down resulting in erasures and a 
loss of information at the receiving side. The receiver knows which subset of the K channels is working; 
the transmitter does not. The problem is then to design an MD system which, for given channel rates, 
minimizes the distortions due to reconstruction of the source using information from any subsets of the 
channels. Currently, the achievable MD rate-distortion region is only completely known for the case of 
two channels, squared-error fidelity criterion and a memoryless Gaussian source [2], [3]. The bounds 
of [3] have been extended to stationary and smooth sources in [4], [5], where they were proven to be 
asymptotically tight at high resolution. Inner and outer bounds to the rate-distortion region for the case 
of if > 2 channels were presented in [6]-[8] but it is not known whether any of the bounds are tight 
for K > 2 channels. 

The earhest practical MD schemes, which were shown to be asymptotically optimal at high resolution 
and large lattice vector quantizer dimensions, were based on the principle of index assignments, cf. [9]- 
[13]. Unfortunately, the existing methods for constructing index assignments in high vector dimensions are 
complex and computationally demanding. To avoid the difficulty of designing efficient index assignments, 
it was suggested in [14] that the index assignments of a two-description system can be replaced by 



DRAFT 



July 15, 2009 



0STERGAARD AND ZAMIR: MULTIPLE-DESCRIPTION CODING BY DITHERED DELTA-SIGMA QUANTIZATION 



3 



successive quantization and linear estimation. More specifically, the two side descriptions can be linearly 
combined and further enhanced by a refinement layer to yield the central reconstruction. The design 



of [14] suffers from a rate 



MD rate-distortion bound 



_ OSS of 0.5 bit/dim. at high resolution and is therefore not able to achieve the 
Recently, however, this gap was closed by Chen et al. [15] who recognized 
that the rate region of the MD problem forms a polymatroid, and showed that the corner points of this 
rate region can be achieved by successive estimation and quantization. The design of Chen et al. is 
inherently asymmetric in the description rate since any corner point of a non-trivial rate region will lead 
to asymmetric rates. To symmetrize the coding rates, it is necessary either to time-share between comer 
points, or to break the quantization process into additional stages, which is a method known as "source 
splitting" (following Urbanke and Rimoldi's rate splitting approach for the multiple access channel). 
When finite-dimensional quantizers are employed, there is a space-filling loss due to the fact that the 
quantizer's Voronoi cells are finite dimensional and not completely spherical, [16], and as such each 
description suffers a rate loss. The rate loss of the design given in [15] is that of 2K — 1 quantizers 
because source sp 
K side quantizers 13 



itting is performed by using an additional K — 1 quantizers besides the conventional 



An interesting open question is: can we avoid both the complexity of the index assignments and the 
extra space-filling loss due to source splitting in symmetricy MD coding? 

Inspired by the works presented in [14], [15], [17], we present a two-channel MD scheme based on two 
times oversampled dithered Delta-Sigma quantization, which is inherently symmetric in the description 
rate and as such there is no need for source splittingO The rate loss when employing finite-dimensional 
quantizers (in parallel) is therefore given by that of two quantizers. The side-to-central distortion ratio 
is controlled by the noise shaping filter; the more "high-pass" the noise is, the larger is the side-to- 
central distortion ratio. Asymptotically as the dimension of the vector quantizer and order of the noise 
shaping filter approach infinity, we show that the symmetric two-channel MD rate-distortion function 
for a memoryless Gaussian source and MSE fidelity criterion can be achieved at any resolution. It is 
worth emphasizing that our design is not limited to two descriptions but, in fact, an arbitrary number 

'The term rate loss refers to the rate redundancy of the specific implementation, i.e. the additional rate required due to using 
a sub-optimal MD scheme. 

^By use of time-sharing, the rate loss can be reduced to that of only K quantizers. Moreover, in the two-description scalar 
deterministic case, the rate loss can be further reduced, cf. [15]. 

^By symmetric we refer to the case where the MD scheme has balanced description rates and balanced side distortions, 
''it should be noted that it is difficult to extend the proposed construction to allow for asymmetric description rates. 
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of descriptions can be created simply by increasing the oversampling ratioj^ However, in this paper, we 
only prove optimality for the case of two descriptions. 

In the Delta-Sigma quantization literature there seems to be a consensus of avoiding long feedback 
filters. We suspect this is mainly due to the fact that the quantization error in traditional Delta-Sigma 
quantization is a deterministic non-linear function of the input signal, which makes it difficult to perform 
an exact system analysis. Thus, there might be concerns regarding the stability of the system. In our work 
we use dithered (lattice) quantization, so that the quantization error is a stochastic process, independent 
of the input signal, and the whole system becomes linear. This linearization is highly desirable, since 
it allows an exact system analysis for any filter order and at any resolution]^ The case of infinite filter 
order has a very simple solution in the frequency domain, which (for large lattice dimension) guarantees 
that the proposed scheme achieves the symmetric two-channel MD rate-distortion function [2], [3]. 

Besides the quantizer-based MD schemes mentioned above there exist several other approaches, e.g. 
MD schemes based on quantized overcomplete expansions [19]-[22]. The works of [19], [20] are based 
on finite frame expansions and that of [21], [22] are based on redundant M-channel filter banks. 

It is well known that there is a connection between quantized overcomplete expansions and Delta- 
Sigma quantization, cf. [23]-[25]. Furthermore, as mentioned above, the connection between overcomplete 
expansions and the MD problem has also been established. Yet, to the best of the authors knowledge, 
none of the schemes presented in [19]-[22] are able to achieve the above mentioned MD rate-distortion 
bounds. Furthermore, the use of Delta-Sigma quantization explicitly for MD coding appears to be a new 
idea. In this paper, we show that traditional Delta-Sigma quantization can be recast in the context of MD 
coding and furthermore, that it provides an optimal solution to the MD problem in the symmetric case. 

The paper is structured as follows: In Section HI! we provide an introduction to dithered Delta-Sigma 
quantization. In Section |llll a connection between Delta-Sigma quantization and MD coding is established 
and we present the main theorem of this work. The proof of the theorem is deferred to Section |Vl 
Section |IV] presents an asymptotic characterization and performance analysis of the proposed scheme in 
the limit of high dimensional vector quantization and high order noise shaping filter. Section |Vl] shows that 
the proposed scheme is, in fact, asymptotically optimal at high resolution for any i.i.d. source with finite 

^When considering more than two descriptions, the distortion generally depends upon the particular subset of received 
descriptions whereas the coding rate is the same for all descriptions. 

^Notice that our results are valid in steady state where the system is time invariant, i.e. we assume the system has been 
operating for a long time so that possible short-time temporal transient effects can be ignored. When referring to variances and 
power spectra we therefore always mean the stationary variances and stationary power spectra [18]. 
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differential entropy. An extension to K descriptions is presented in Section IVlIl and finally, Section IVIIII 
contains the conclusions. 

II. Dithered Delta-Sigma Quantization 

Throughout this paper we will use upper case letters for stochastic variables and lower case letters 
for their realizations. Infinite sequences and L-dimensional vectors will be typed in bold face. We let 
X ~ A^(0, a\) denote a zero-mean Gaussian vaiiable of variance u^, and X = {Xi,X2, . . . } denote an 
infinite sequence of independent copies of X. Thus X is an i.i.d. (white) Gaussian process. Moreover, 
X = {xi, X2, • • • , } denotes a realization of X where is the /cth symbol of x. 

A. Preliminaries: Entropy-Coded Dithered Quantization 

Z Z 

- s 

) ' 



Fig. 1. Entropy-constrained dithered (lattice) quantization (ECDQ). The dither signal Z is assumed known at the decoder. 
The quantizer Ql is an L-dimensional lattice vector quantizer and the rate of the entropy coder is given by the entropy of the 
quantized output of Ql conditioned upon Z. 
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Before introducing our dithered Delta-Sigma quantization system, let us recall the properties of entropy- 
coded dithered (lattice) quantization (ECDQ) [26]. ECDQ relies upon subtractive dither; see Fig. [T] For 
an L-dimensional input vector S, the ECDQ output is given by S = Ql{S + Z) — Z, where Ql denotes 
an L-dimensional lattice quantizer with Voronoi cells [27]. The dither vector Z, which is known to both 
the encoder and the decoder, is independent of the input signal and previous realizations of the dither, and 
is uniformly distributed over the basic Voronoi cell of the lattice quantizer. It follows that the quantization 
error 

E = S-S = Ql{S + Z)-S-Z (1) 

is statistically independent of the input signal. Furthermore, E is an i.i.d.-vector process, where each 
L-block is uniformly distributed over the mirror image of the basic cell of the lattice, i.e., as —Z. In 
particular, it follows that £ is a zero-mean white vector with variance a"^ [26], [28]. 

The average code length of the quantized variables is given by the conditional entropy H{Ql{S+Z)\Z) 
of the quantizer Ql, where the conditioning is with respect to the dither vector Z. It is known that this 
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conditional entropy is equal to the mutual information over the additive noise channel Y = S + E where 
E (the channel's noise) is distributed as —Z; see [26] for details. The coding rate (per L-block) of the 
quantizer is therefore given by 

H{Ql{S + Z)\Z) = I{S; Y) = h{S + E) - h{E) (2) 

where /(•, •) denotes the mutual information and h{-) denotes the differential entropy. If subsequent 
quantizer outputs are entropy-coded jointly, then we must change the blockwise mutual information 
in the rate formula Q to the joint mutual information between input-output sequences (if there is no 
feedback) [26], or to the directed mutual information (if there is feedback) [29], [30]. 

If the source S is white Gaussian, then the coding rate Q, normalized per-sample, is upper bounded 

by 

\H{QUS + Z)\Z)<\\og, + 



+ ilog2(27reGL) 

= Rs{D) + ^log2{27TeGL) (3) 

where Gl is the dimensionless normaUzed second moment of the L-dimensional lattice quantizer Ql [27]. 
In the second equality, D is the total distortion after a suitable post-filter (multiplier) and Rs{D) is the 
rate-distortion function of the white Gaussian source S; see [31]. The quantity lireGL is the space-filling 
loss of the quantizer and i log2(27reGL) is the divergence of the quantization noise from Gaussianity. It 
follows that it is desirable to have Gaussian distributed quantization noise in order to make as small 
as possible and thereby drive the rate of the filtered quantizer towards Rs{D). Fortunately, it is known 
that there exists lattices where Gl l/27re as L ^ cxd; the quantization noise of such quantizers is 
white, and becomes asymptotically (in dimension) Gaussian distributed in the divergence sense [28]. 

B. Delta-Sigma ECDQ 

We are now ready to introduce our dithered Delta-Sigma quantization system. We begin with the 

Fig. dz ■ 



single description case which is sketched in 



The multiple descriptions case will be discussed in 



Section |llll The input sequence x is first oversampled by a factor of two to produce the oversampled 
sequence a. It follows that a is a redundant representation of the input sequence x, which can be obtained 

'The Delta-Sigma quantization system shown in Fig.|2]is a discrete-time version of the general noise-shaping coder presented 
in [32]. The system has an equivalent form where the feedback is first subtracted and this difference is then filtered [32]. 
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Fig. 2. Dithered Delta-Sigma quantization. 



simply by inserting a zero between every sample of x and applying an interpolating (ideal lowpass) filter 
h{z). For a wide-sense stationary input process X, the resulting oversampled signal A would be wide- 
sense stationary, with the same variance as the input, and the same power-spectrum only squeezed to 
half the frequency band as shown in Fig. [3] In particular, a white Gaussian input becomes a half-band 
low-pass Gaussian process with 



Var(Afc) = Var(Xfc) = a\. 



(4) 



At the other end of the system we apply an anti-aliasing filter ha{z), i.e. an ideal half-band lowpass 



A 



TT — 7r/2 n/2 tt lo 

(a) Spectrum of X (b) Oversampling by two 




— TT — 7r/2 7r/2 tt lo 
(c) Spectrum of A 

Fig. 3. The power spectrum of (a) the input signal and (c) the oversampled signal, (b) illustrates the oversampling process 
where the input signal is first upsampled by two and then filtered by an ideal half-band lowpass filter. 



filter, and downsample by two in order to get back to the original sampling rate. 

We would like to emphasize that the dithered Delta-Sigma quantization scheme is not limited to 
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oversampling ratios of two. In fact, arbitrary (even fractional) oversampling ratios may be used. This 
option is discussed further in Section IVIII 

The oversampled source sequence a is combined with noise feedback e, and the resulting signal a' 
is sequentially quantized on a sample by sample basis using a dithered quantizer. For the simplicity of 
the exposition we shall momentarily assume scalar quantization, i.e., L = 1. The extension to L > 1 is 
discussed in Section III-CI The quantization error of the kth sample, given for a general ECDQ by 
([D, is fed back through the (causal) filter c'{z) = J2^=i c-iz~^ and combined with the next source sample 
afe_|_i to produce the next ECDQ input a'^^^^. Thus, the output of the quantizer can be written as 

Ofc = a'k + Ck = ttk + Bk + Bk = ak + ek (5) 
where e{z) = c'{z)e{z) or equivalently 

p 

Cfc — ^ ^ Ci&k—i- 
1=1 



dk 





t2 
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ak 
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Fig. 4. The dithered quantizer is replaced by the additive noise model. 

As explained above, the additive noise model is exact for ECDQ and we can therefore represent the 
quantization operation as an additive noise channel, as shown in Fig. |4l In view of this linear model, the 
equivalent reconstruction error in the oversampled domain, denoted in (|5]l, is statistically independent 
of the source. Thus we call the "equivalent noise". Notice that is obtained by passing the quantization 
error through the equivalent pth order noise shaping filter c{z), 

c{z)^j2'^^'~' (6) 

i=0 

where cq = 1 so that c{z) = I + c'{z). Since the quantization error e of the ECDQ ([T|| is white with 
variance a'^, it follows that the equivalent noise spectrum is given by 



= \c{en\'cTl 
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The fact that the output is obtained by passing the quantization error through the noise shaping 
filter c{z) and adding the result to the input au can be illustrated using an equivalent additive noise 
channel as shown in Fig. |5l 



efe 
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h{z) 











c(z) 



) ^ 


ha{z) 




\2 











Fig. 5. The equivalent additive noise channel: The output tik is obtained by passing the quantization error Ck through the noise 
shaping filter c{z) and adding the result to the input ak- 



We may view the feedback filter c'{z) as if its purpose is to predict the "in-band" noise component 
of efc based on the past p quantization error samples efc_i,efc_2, • • • ,e.k-p (at the expense of possibly 
increasing the "out-of-band" noise component). The end result is that the equivalent noise spectrum (|7]) 
is shaped away from the in-band part of the spectrum, i.e., from the frequency range (— 7r/2, +7r/2), as 
shown in Fig. |6] Notice that due to the anti-aliasing filter ha(z), only the in-band noise determines the 
overall system distortion. The exact guidelines for this noise shaping are different in the single- and the 
multiple description cases, and will become clear in the sequel. 



Sa 



■2ul 




Fig. 6. Illustrated on the left the case where there is no feedback and the quantization noise is therefore flat (in fact white) 
throughout the entire frequency range. On the right an example of noise shaping is illustrated. The grey-shaded areas illustrate 
the power spectra of the noise and the hatched areas illustrate the power spectra of the source. 



As previously mentioned, if we encode the quantizer output symbols independently, then the rate R of 
the ECDQ is given by the mutual information between the input and the output of the quantizer. Thus, 
the rate (per sample) is given by 

R = I{A'^-Ak)=I{A'j,-A'^ + Eu) (8) 
where Ei^ is independent of the present and past samples of yl'^ by the dithered quantization assumption. 
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If Ak and were Gaussian, as discussed in Section III-CI below, then we could get 

-^'-(-^) 

where Var(A'^) denotes the variance of the random variable A'f^. At high resolution conditions the variance 
of the error signal e (and therefore of e) is small compared to the source, so by ^ we have Var(A'^) w o"^ 
which implies that ^ becomes 

R»ilog,(|) (10) 

where « in (fTOl) is in the sense that the difference between both sides of the equation goes to zero as 
a"^ — > 0. We can now combine ([TOl l with the expression ^ for the noise spectrum to obtain a simple 
overall rate-distortion characterization of the system. It can be observed that the resulting R{D) curve 
depends on both the in-band and the out-of-band noise components. 

If we apply joint entropy coding of the quantizer outputs, that is, we let the entropy coder take advantage 
of the memory inside the oversampled source, then the rate of the Delta-Sigma quantization scheme is 
independent of the out-of-band noise spectrum. To see this, recall that for jointly-coded ECDQ within a 
feedback loop, the coding rate is given by the directed mutual information rate, that is, [30], 

I{Ai^A', + Ek) 

= I{A',;Ai + Ek\A',_, + Ek-i,A',_^ + Ek^2, • • • ) 
= + Ek\A'j^_^ + Ek-i,A'i^_2 + Ek^2, ...)- h{Ek) 
= h{Ak + e/cl^fe-i + ek-i,Ak-2 + ek-2, ■■■) - h{Ek) 
h{A + e) - h{e) 

= I{A-A + e) (11) 

where h{-) and /(•) denote the entropy rate and mutual information rate, respectively. In the equations 
above (a) follows since A'^ = A^ + Ek and = E^ + Ek- In (6) we used the fact that E^ is the 
prediction error of given its past so that h{Ek) is the entropy rate of e, i.e. h{e) = h{E}S) = h{Ek). 
Asymptotically as L ^ oo, the quantization noise becomes approximately Gaussian distributed, and the 
equivalent ECDQ channel in Fig. |4] is AWGN (see Section III-CI below). Recall that, for a Gaussian 
process, disjoint frequency bands are statistically independent. Therefore, since the input A is lowpass, 
the mutual-information rate (fTTI ) is independent of the out-of-band part of the noise process e. Thus, the 
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joint-entropy coding rate is independent of the out-of-band noise spectrum^ 

C. Vector Delta-Sigma Quantization 

To justify the use of high-dimensional vector quantizers we will consider a setup involving L indepen- 
dent sources ^ These sources can, for example, be obtained by demultiplexing the original memoryless 
process X into L independent parallel i.i.d. processes X^'-^ = {X„2,+i},Vn G Z and I = 1,... ,L\^ 
In this case the nth sample of the Ith process X^'-^ is identical to the {n x L + l)th sample of the 
original process X. Let us give an example where L = 3 so that we have three processes X^^\X^'^\ and 
The three processes are each upsampled by a factor of two so that we obtain the three processes 
A^^\A^'^\ and A^^\ where each is input to a Delta-Sigma quantization system as shown in Fig. |7J Hence, 
in this case, three coders are operating in parallel and instead of a single sample a'^ we have a triplet 
of independent samples {a^^\a^'^\a'^^^). This makes it possible to apply three-dimensional ECDQ on 
the vector formed by cascading the triplet of scalars. If L coders are operating in parallel, we can form 
the set of L independent samples {a'l^^ , , • • • , aj^^'' ) and make use of L-dimensional ECDQ on the 
vector {al:^\ ■ ■ ■ , (^k^^)- In general, we will allow L to become large so that, according to ([3]) and 
the paragraph that follows just below Q, the rate loss i log2(27reGL) due to the quantization noise being 
non-Gaussian can be made arbitrarily small. Thus, for large L, in ([8]) can indeed be approximated as 
Gaussian noise. 



III. Multiple-Description Coding 
A. MD Delta-Sigma Quantization 

In this section we show that the sequential dithered Delta-Sigma quantization system, which is shown 
in Fig. |2j can be regarded as an MD coding system. For example, in the case of an oversampling ratio 
of two, each input sample leads to two output samples and we have in fact a two-channel MD coding 
system as shown in Fig. [8] 

In the MD scheme of Fig. [H the first description is given by the even outputs of the lattice quantizer and 
the second description by the odd outputs. Each description is then entropy-coded separately, conditioned 

^Interestingly as we shall see later, in the MD case the coding rate depends on the in-band as well as the out-of-band noise 
spectra; see i26t . 

' The idea of applying lattice ECDQ to feedback coding systems in parallel was first presented in [30]. 
'"Notice that the delay between two consecutive samples of the Zth process will be that of L input samples. 
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(a) Demultiplexing the i.i.d. source into 
L — 3 independent streams 
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(b) Applying a three-dimensional lattice quantizer Qs 



Fig. 7. The dashed box illustrates that the triplet of scalars (qj. 



/(I) H2) ,(3) 



) are jointly quantized using three-dimensional 



ECDQ. Notice that we may see the three-dimensional lattice quantizer Q3 as a composition of three functions where a 



(1) 



and a^^' = 



upon its own dither, and transmitted to the decoder. Notice that although the oversampled signal A has 
memory, the source part in each description is memoryless, because we assume ideal interpolation so for 
a Gaussian source the even/odd splitting of the samples corresponds to downsampling by two. However, 
unless the shaped and aliased noise is white and Gaussian, there will be memory in the downsampled 
signal Aeven or Aodd- We show later that, asymptotically as the vector dimension of the quantizer and the 
order of the noise-shaping filter approach infinity, the downsampled noise becomes an i.i.d. process, and 
entropy coding can therefore be done sample-by-sample (i.e. memorylessly) without loss of optimality. 
By Q, the sample -by-sample ECDQ rate is given by the (per-sample) block-wise mutual information 

R= yliA'-A' + E). (12) 
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At the decoder, if both descriptions are received, then they are interlaced to form back the oversampled 
signal A, an anti-aliasing filter ha{z) (i.e. an ideal half -band lowpass filter) is applied and the signal is 
then downsampled by two and scaled by (3 as shown in Fig. |9] If only the even samples are received, 
we simply scale the signal by a. On the other hand, if only the odd samples are received, we first apply 
an all-pass filter hp{z) to correct the phase of the second description and then scale by a. The all-pass 
filter hp{z) is needed because the upsampling operation performed at the encoder, i.e. upsampling by 
two followed by ideal lowpass filtering (sinc-interpolation), shifts the phase of the odd samples. The post 
multipliers a and f3 are described in Section IIV-CI 
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Fig. 8. Two-channel MD coding based on dithered Delta-Sigma quantization: Encoder. 
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Fig. 9. Two-channel MD coding based on dithered Delta-Sigma quantization: Decoder. 



The distortion due to reconstructing using both descriptions is traditionally called the central distortion 
dc and the distortion due to reconstructing using only a single description is called the side distortion dg. 

B. The Symmetric MD Rate-Distortion Region 

We will need the solution to the quadratic (memory less) Gaussian MD problem, as proven by Ozarow [3], 
in the symmetric case, i.e., when both descriptions have the same rate R and the side distortions are equal. 
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The set of achievable distortions for description rate R is the union of all distortion pairs {dc, ds) satisfying 

ds > ai2-2« (13) 

and 

> (14) 

i-(^/n-^/A)2 

where 11 = (1 — ds/a\)'^ and A = d^g/a\^ — 2~^^ and where we require 11 > A to avoid degenerate 
cases. 

Based on the results of [3], it was shown in [33] that at high resolution, for fixed central-to-side 
distortion ratio dc/dg, the product of the central and side distortions of an optimal two-channel MD 
scheme approaches 

4 1 - dc/ds 

where the approximation = here is in the sense that the ratio between both sides goes to 1 as ^ 
(or R —>■ oo). If ds/dc ^ 1, i.e., at high side -to-central distortion ratio, this simplifies to 

d,ds = ^2-'^. (16) 



C. Main Theorem 

We now present the main theorem of this work, which basically states that the MD Delta-Sigma 
quantization scheme (presented in Section ITlI- Al l can asymptotically achieve the lower bound of Ozarow's 
MD distortion region (presented in Section IIII-BI ). 

Theorem 1: Asymptotically as the noise-shaping filter order p and the vector-quantizer dimension L 
are going to infinity, the entropy rate and the distortion levels of the dithered Delta-Sigma quantization 
scheme (of Figs. [8] and |9l) with optimum filters and lattice quantizer achieve the symmetric two-channel 
MD rate-distortion function (fT3l ) - (fT4l) for a memoryless Gaussian source and MSB fidelity criterion, at 
any side-to-central distortion ratio dg/dc and any resolution. Furthermore, the optimal infinite-order noise 
shaping filter is unique, minimum phase, and its magnitude spectrum |c(e-''^)| is piece-wise flat with a 
single jump discontinuity at cj = 7r/2. 

Before presenting the proof of the theorem, we provide in the following sections a series of supporting 
lemmas. The proof of the theorem can be found in Section IVl 
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IV. Asymptotic Characterization and Performance Analysis 

In this section we concentrate on the asymptotic case where p,L^oo, i.e. infinite noise shaping filter 
order and infinite vector quantizer dimension. For analysis purposes, this allows us to assume Gaussian 
quantization noise in the system model of Fig. |4l with arbitrarily shaped equivalent noise spectrum ([T]). 

A. Frequency Interpretation of Delta-Sigma Quantization 

We first give an intuitive frequency interpretation of the proposed Delta-Sigma quantization scheme. 
This frequency interpretation reveals that the role of the noise shaping filter is not simply to shape away 
the quantization noise from the in-band spectrum, as is the case in traditional Delta-Sigma quantization, 
but rather to delicately control the tradeoff between the in-band noise versus the out-of-band noise, which 
translates into a tradeoff between the central and side distortions. This tradeoff is done while keeping 
the coding rate fixed, which, at least at high resolution, is equivalent to keeping the quantizer variance 
cr|; fixed. See ( fTOl ). 

Recall that we, at the central decoder, apply an anti-aliasing filter (ideal lowpass filtering) before 
downsampling. Hence, the central distortion is given by the energy of the quantization noise that falls 
within the in-band spectrum. The inclusion of a noise shaping filter at the encoder makes it possible to 
shape away the quantization noise from the in-band spectrum and thereby reduce the central distortion. By 
increasing the order of the noise shaping filter it is possible to reduce the central distortion accordingly. 

It is also interesting to understand what influences the side distortion. The side descriptions are 
constructed by using either all odd samples or all even samples of the output A. Hence, we effectively 
downsample ^ by a factor of two. It is important to see that this downsampling process takes place 
without first applying an anti-aliasing filter. Thus, aliasing is inevitable. It follows, that not only the 
noise which falls within the in-band spectrum contributes to the side distortion but also the noise that 
falls outside the in-band spectrum (i.e. the out-of-band noise) affects the distortion. Since, in traditional 
Delta-Sigma quantization, the noise is shaped away from the in-band spectrum as efficiently as possible, 
the out-of-band noise is likely to be the dominating contributor to the side distortion. We have illustrated 
this in Fig. \M 

It should now be clear that, in two-channel MD Delta-Sigma quantization, the role of the noise shaping 
filter is to trade off the in-band noise versus the out-of-band noise. In particular, in the asymptotic case 
where the order of the noise shaping filter goes to infinity, it is possible to construct a brick-wall filter 
which has a squared magnitude spectrum of 1/6 in the passband (i.e. for |a;| < it/2) and of 6 in the 
stopband (i.e. for 7r/2 < \lo\ < vr). In this case, the central distortion is proportional to 1/6 whereas 
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Fig. 10. The power spectrum of (a) the quantization noise (b) the shaped quantization noise. In (b) the energy of the lowpass 
noise spectrum (the bright region) corresponds to the central distortion and the energy of the full spectrum corresponds to the 
side distortion. 



the side distortion is proportional to 1/5 + 5. This situation, which is illustrated in Fig. |10(b)[ will be 
discussed in more detail in the next section. 



B. Achieving the MD Distortion Product at High Resolution 

It is possible to take advantage of the frequency interpretation given in Section IIV-AI in order to show 
that the optimum central-side distortion product at high-resolution (fTSl) can be achieved by Delta-Sigma 
quantization. We later extend this result and show that with suitable post-multipliers at the decoders, 
optimum performance is achieved at any resolution. 

Lemma 1: At high resolution and asymptotically as jj, L ^ oo, the distortion product given by (fTSl) 
is achievable by Delta-Sigma quantization. 

Proof: The central distortion is equal to the total energy cr'^Pd^ of the in-band noise spectrum where 



tt/2 



-tt/2 



(17) 



The side distortion is equal to the energy cr'^Pd, of the in-band noise spectrum of the side descriptions 
which contains aliasing due to the subsampling process. Since we downsample by two we have 

Let us shape the noise spectrum as illustrated in Fig. |10(b)l Thus, we let |c(e'''^)p = 1/(5 for |w|<7r/2 
and |c(e-^''^)p = 5 for -k/2 < < vr where < 5 G M. It follows from ^TS^ that, for any 5 > 0, 



,(ei-/2)|2 + |c(eJ(W2+-))|2rf^. 



(18) 
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4 = + (5-1) and from (O we see that dc = ^c"!;/^ which yields the distortion product 

dcds = ^^^^i (19) 

From (fTOl) we know that at high resolution R « ^ log2(cr^/o"|;) (where ^ is in the sense that the 
difference goes to zero as R ^ oo), so that 

4 = 42-4^ (20) 

(where = is in the sense that the ratio goes to one as i? ^ oo). Finally, since dc/ds = S^^ / {6 + 6^^) it 
follows that 

' (21) 



1- dc/ds 5 

and the lemma is proved by inserting (l20l ) and (|2TI ) into ([T9l ) and comparing the resulting expression 
to (O. ■ 

C. Optimum Performance for General Resolution 

In this section we extend the optimality result of Section IIV-BI above, and show that the two-channel 
Delta-Sigma quantization scheme achieves the symmetric quadratic Gaussian rate-distortion function at 
any resolution. 

Let Ui denote the reconstructions before the side post multipliers so that Xi = aUi,i = 0, 1, and let E 
denote the expectation operator. It can then be shown that EXUi = a\ and WJ^ = a\ + a'^E{5 + 5~'^)/2. 
Moreover, let U denote the reconstruction before the central multiplier {3. Then Ef7^ = a\ + a\5~^ /2. 
Finally, let the post multipliers be given by 

a- ^ 



and 

/3 



•X "T 

It follows that the side distortion is given by 



ds = E(X, - Xf 
= E(aC/i - xf 

= a\-2aa\ + a^{a\+al{5 + 5-^)/2) (22) 
44(5 + 5-1) 



2al + al{5 + 5-^] 



(23) 
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Similarly, let Xc = PU so that the central distortion is given by 



E{PU-X) 



2 



+ P'i'^x + (^E^-'m - Wtx (24) 
^ ^ (25) 



2a\ + cj|(^-i ■ 



Lemma 2: For a given description rate R and asymptotically as p, L — > oo (i.e., assuming Gaussian 
quantization noise and equivalent noise spectrum as in Fig. |10(b)| ), the side distortion given by ( |23] ) and 
the central distortion given by ( |25l) achieve the lower bound (fT4l ) of Ozarow's symmetric MD distortion 
region. 

Proof: Recall from Section |II1 that the rate of memoryless-ECDQ (assuming that the entropy coding 
is conditioned upon the dither signal and that the dither signal is known at the decoder) is equal to the 
mutual information between the input and the output of an additive noise channel (fT2l) . For large L, this 
mutual information can be calculated as if the additive noise E}^ was approximately Gaussian distributed. 
It thus follows from ([8]l and Q that as L ^ cxd the description rate becomes 

R = I{A'^-Ak) 
= h{Ak) - h{Ek) 

= \ log2(2^e(ai + 4(5 + r i)/2)) - \ \og^{27Teal) 



1 log + 

2 a 



'E 



log2 ( ' "^^_V " ) . (26) 



We can rewrite (l26l) as 



2-4« = (27) 



By use of (l23j) and (E?]) we then get 

a%{5^ - 25^ + 1) 



and 



so that 



A 



n 



{2al5 + al5^ + alf 
{2a\5 + al5^ + alY 
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Finally, inserting (1281 ) in (fT4l) leads to 



a\2- 



4R 



1 - (\/n - yA)2 2a\6 + al 
which is identical to (l25l ) and therefore proves the lemma. 



D. Relation to Ozarow's Double Branch Test Channel 

Let us now revisit Ozarow's double branch test channel as shown in Fig. [TT] In this model the noise 
pair {Nq,Ni) is negatively correlated (except from the case of no-excess marginal rates, in which case 
the noises are independent). Notice that this is in line with the above observations, since the highpass 

No 

i Un "0 - 



X 



Xr 



t 



Ui 



Fig. 11. The MD optimum test channel of Ozarow [3]. At high resolution Qi = 1 and Pi — 1/2, i = 0,1 so that Xo ~ 

Uo,Xi = Ui and Xc = + Xi). 



nature of the noise shaping filter causes adjacent noise samples to be negatively correlated. The more 
negatively correlated they are, the greater is the ratio of side distortion to central distortion. Furthermore, 
at high resolution, the "filters" (a-i and = 0,1) in Ozarow's test channel become trivial and the 
central reconstruction is simply given by the average of the two side channels. This averaging operation 
can be seen as a lowpass filtering operation, which leaves the signal (since it is lowpass) and the in-band 
noise intact but removes the out-of-band noise. 

More formally, for the symmetric case (where aj^ = cr^ , i = 0, 1 and p is the correlation coefficient 
of the noises), we have the following high-resolution relationships between {p,afj) of Ozarow's test 
channel and {6,a'^) of the proposed Delta-Sigma quantization scheme. 

Lemma 3: At high-resolution conditions, we have 

ct| = a%V^ - (29) 
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and 



yi - P 

6 = ^==. (30) 



Proof: From [4], [5] it follows that Ozarow's sum rate Rq + Ri satifies 

Ro + Ri> liX; X + No)+ I{X; X + N^) 
+ IiX + No;X + Ni) 
= I{X- X + N^)+ I{X- X + Ni)+ I{No; N^] 
= I{X; X + No)+ I{X; X + Ni) 



h{X + iVo) - + Nq\X) + + iVi 



-/i(X + iVi|X)-log2(VW 



,2^ 



N 



logaR^^ -log2 Vl-P^ (31) 



where the last equality follows since the noises have equal variances. By equating (1311 ) to (1261 ). i.e. 

2R = \og2{ '^^^'^'^^^'i^^ — ^) and solving for a"^^ we obtain 



MMSE = ^ . (33) 



If a^j <C (T^, this reduces to al^ cr^ \J\ — and we obtain (l29l ). 

The MMSE when estimating X from two jointly Gaussian noisy observations f/j = X + A'^j, i = 0, 1 
(where the Gaussian noises have equal variances), is given by 

a2,(l + /5) + 2 

Thus, the central distortion of Ozarow's test channel is given by (1331 ). which we equate to the central 
distortion (|25]) . solve for b and insert a\ from (l32l) . that is 

^_ aiK(l + />) + 2) yW 

2(ai+a^)-(5 + 5-i)^^yr^ ^1+^ 



2(4 + ^^)-(^ + ^-'K\/i^ 



Once again, letting o^q <^ a\ it foUows that 5 ^ /j— ^ , which yields (l30l ) and thereby proves the lemma. 



Remark 1: The relationship between Ozarow's test-channel and the Delta-Sigma quantization scheme 
at general resolution is provided by (l32l ) and (l34l ). 
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V. Proof OF Theorem [I] 

We are now in a position to wrap up the proof of Theorem [U Lemma factually shows that it is possible 
to achieve the quadratic Gaussian rate-distortion function if we replace the ECDQ by a Gaussian noise, 
and the equivalent noise spectrum dTJl by a brick wall spectrum. This can be viewed as setting the lattice 
quantizer dimension L and the feedback filter order p to be equal to infinity. Thus, what is stiU missing is 
the characterization of the limit behavior of the coding rate as L,p ^ oo, and the distortion as p ^ oo. 



A. Distortion loss 

We first present Lemma |4] (with a proof in the appendix) which describes the central and the side 
distortions at general resolution when using an arbitrary noise-shaping filter c{e^^). 

Lemma 4: For any given pth-order noise-shaping filter c{e^'^) and optimal multipliers (a and the 
central distortion is given by 

4 = (35) 



and the side distortion is given by 



(36) 



where 



and 



Pd = — I \c{en?doo = -yy 



p p 

sine I I c, c 



j=0 j=0 



A 

The noise shaping filter used in the proof of Lemma |2] to show achievability of the quadratic Gaussian 
rate-distortion function is of infinite order, and it satisfies 

^ j\og^\c{en?dw = Q. (37) 

It follows that the area under log2 |c(e-^'^)p is equally distributed above and below the dB line, which 
is a unique property of minimum-phase filters [34]. In fact, the following Lemma proves that, in order 
for c{z) to be optimal, it must be of infinite-order and minimum phase (see the appendix for the proof). 
This means that the optimum noise shaping filter is unique. 

Lemma 5: In order to achieve the quadratic Gaussian rate-distortion function, it is required that the 
noise shaping filter c(z) is of infinite order, minimum-phase, and have a piece-wise fiat power spectrum 
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of power 6 ^ in the lowpass band (i.e. for |u;| < tt/2) and of power 5 in the highpass band (i.e. for 



Remark 2: It can be observed from (l30b that (at high resolution) choosing 6 in the range < 6 < 
1 corresponds to having positively correlated noises in Ozarow's test-channel. Moreover, the central 
distortion is proportional to 1/6 whereas the side distortion as well as the description rate are both 
proportional to the sum 6 + 1/6. Thus, while it is not a system requirement to choose 5 > 1, it is a better 
choice performance wise. 

We now assess the distortion loss due to using a finite order noise-shaping filter. Let S^^{uj) = 
a'^\c°'f^{e^'^)\'^ denote the power spectrum of the shaped noise when using the ideal infinite-order noise 
shaping filter c°^^{e^'^), which is optimal and unique as proven by Lemma |5l Thus, \c°^^{e^^)\'^ is piece- 
wise fiat with a jump discontinuity at cj = 7r/2, cf. Fig. |10(b)| For such a function, point- wise convergence 
of the Fourier coefficients cannot be guaranteed. However, we do have convergence in the mean square 
sense [35]. Specifically, let s'f\uj) denote the pth order Fourier approximation to S°'^^{uj). Then [35] 



which asserts that the limit for p ^ cxd exists. In addition, it can be shown that the error (MSE) of the 
pth order Fourier approximation of this step function is of the order 0{l/p) [36]. It follows that for any 
p we have 



7r/2 < < vr) where 1 < 5 G M. 



A 




(38) 



a\al{P,^+0{l/v)) 



a\ + al{P,^ + 0{l/p)y 



(39) 



a\al{P,^ + 0{l/p)) 



a\ + al{Pd,+0{l/p)y 



and the desired continuity in the limit p ^ oo is established. 



B. Coding rate loss 



By similar arguments as leading to ( [39l) . for a finite p, the variance of A}^ is given by 



Var{Ak) = E[(A'fc + E^f] 

= al + al{Pds + 0{l/p)) 



(40) 



(41) 



Moreover, the coding rate ([S]) when using memoryless entropy coding is given by 



= h{Ak) - KEl) + Uog^{GL2T^e) 
= h{Au)-h{El) + 0{\og^{L)/L) 



(42) 



(43) 
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where h{Ak) is an increasing function in Var{Ak) and denotes a Gaussian variable (process) with the 
same variance (spectrum) as Ek- Eq. ((42)) follows from the discussion after Q where it may be noticed 
that the term ^ log2(G'L27re) describes the divergence of the quantization noise from Gaussianity; see 
also [31]. This divergence term corresponds to an excess rate due to using a finite dimensional lattice 
quantizer and may be upper bounded by 0(log2(-^^)/-^^) when optimal L-dimensional lattice quantizers 
are used, see [28] for details. Thus, if we keep a'^ fixed, then the coding rate is increased due to the 
0{l/p) variance increase given in (|4T] ) and due to the excess term 0(log2(L)/L) in (1431 ). These rate 
penalties vanish as p, L oo and the desired convergence in coding rate is proved. 

In order to complete the proof of the theorem, we need to show that an optimal monic minimum phase 
filter always exists for any ratio of PdjPda- Towards that end, we keep the post multipliers fixed and 
define J = XcPd^+^sPd, as the cost function to be minimized by the pth-order noise-shaping filter. Notice 
that if we let As = we are only concerned about minimizing the noise power than falls in the in-band 
region. Thus, we are aiming at minimizing the central distortion. On the other hand, letting Ac ^ A^ gives 
priority to the side distortion since the total noise power is minimized. Let cq — 1 and c — (ci, • • • , Cp) 
be the filter coefficients. Moreover, let g be the p-vector with elements gi = sinc(i/2), i = 1, . . . ,p, and 
let G be the p autocorrelation matrix with elements Gi,j = sinc((z — j)/2), where i,j G {1, . . . 
With this it follows that 

p 

Pd.=Y.''} = {l + c'c) 

and 

p p 

I p p p 

= -(l + 2^sinc(V2)Q +^^sinc((i-i)/2)ciCj) 

1=1 i=l j=l 

so that 

AePd^ + A.Prf, = ^ (Ae(l + 2c^g + JOc) + 2As(l + c^c)) 

= \{^c + 2Xs + 2X^Jg + c^(AcG + 2XJ)c) . 

The optimal filter coefficients are found by solving the differential equation -§^{XcPd^ + XsPd^) = 0, that 
is 

c=-{G + 2^I)-^g (44) 
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where / is the p x p identity matrix. Notice that G + 2^/ is a symmetric and full rank matrix and (l44l) 
therefore defines a well-posed problem. The solution to (|44)) can be found by the Yule -Walker method, 
which yields a unique minimum-phase filter [37]. As p ^ oo, the autocorrelation sequence of the impulse 
response of the obtained filter c{z) becomes identical to the ideal autocorrelation sequence whose Fourier 
transform describes the optimum shaped noise spectrum [37]. Thus, the resulting spectrum of the shaped 
noise becomes identical to the optimum spectrum. This proves the theorem. 

VI. Universality of Dithered Delta-Sigma Quantization 

In this section we discuss the universality of the proposed scheme at high resolution. First, notice 
that the central and side distortions depend only upon the second-order statistics of the source and 
the quantization noise, i.e. and a^, and as such not on the Gaussianity of the source. Second, 
independent of the source distribution, the distribution of the quantization noise becomes approximately 
Gaussian distributed (in the divergence sense) in the limit of high vector quantizer dimension L. Finally, 
the ECDQ is allowed to encode each description according to its entropy. Thus, the coding rate is equal 
to the mutual information ([T2l) of the source over the Gaussian test channel. For memoryless sources of 
equal variances, this coding rate is upper bounded by that of the Gaussian source. Moreover, Zamir proved 
in [4] that Ozarow's test channel becomes asymptotically optimal in the limit of high resolution for any 
i.i.d. source provided it has a finite differential entropy. Thus, since the dithered Delta-Sigma quantization 
scheme resembles Ozarow's test channel in the limit as p, L — > cxd, we deduce that the proposed scheme 
becomes asymptotically optimal for general i.i.d. sources with finite differential entropy. 

A delicate point to note, though, is that due to the sine interpolation, the odd samples might not be 
i.i.d. and joint entropy coding within the packet is necessary in order to be optimal. Specifically, with 
joint entropy coding the rate is given by the directed mutual information formula ([TT]) applied to the 
sub-sampled source Af^ odd- The resulting rate for the odd packet is h{Ak^odd) — h{EjS), which (at high 
resolution) is ^ h{X) — ^ log2(27recr|;), as desired [4]. 

If we have a source with memory, and we allow joint entropy coding within each of the two packets, 
then a similar derivation shows that we would achieve rate R « h{X) — ^ log2(27recr|;) in each packet. 
This rate is the mutual information rate of the source over the Gaussian test channel. Since Ozarow's 
test channel is asymptotically optimal in the limit of high resolution for any stationary source with finite 
differential entropy rate, [5], it follows that the proposed scheme is asymptotically optimal for such 
sources as well. It is also interesting to note that we recently proposed to combine noise-shaping with 
prediction when encoding sources with memory [38]. In the Gaussian case, this makes it possible to 
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remove the source redundancy without requiring entropy coders with memory. 

VII. Extension to K > 2 descriptions 

We end this paper by presenting a straight-forward extension of the proposed design to K descriptions, 
though without any claim of optimaUty. The basic idea is to change the oversampUng ratio from two to K 
and then decide which output samples should make up a description^ When dealing with K descriptions, 
2^ — 1 distinct subsets of descriptions can be created. Thus, the design of the decoders is generally more 
complex for greater K. For example, if two out three descriptions are received, aliasing is unavoidable 
(as was the case for K = 2 descriptions). Moreover, due to the fractional (non-uniform) downsampling 
process, the simple brick-wall lowpass filter operation is not necessarily the optimal reconstruction rule. 
In fact, the optimal reconstruction rule depends not only upon the number of received descriptions but 
(generally) also upon which descriptions are received. However, in this section we will restrict attention 
to cases leading to uniform sampling!^ Thus, the design of the decoders is simplified. 

We use the previously presented Delta-Sigma quantization scheme (of Figs. [8] and |9l) but oversample 
now by K instead of two. More specifically, let us assume that K = A and that every fourth sample 
make up a description. We notice that the extension to an arbitrary number of descriptions is straight 
forward. We consider only the cases that leads to uniform (non-fractional) downsampling, i.e. reception 
of any single description, every other description (i.e. two out of four), or all four descriptions. 

It can easily be seen that if we receive all four descriptions, the central distortion dc is given by the 
noise that falls within the in-band spectrum. In other words, 

dc = — S,{uj)dLO (45) 

J-tt/4 

where S^{uj) = \c{e^'^)\'^a'^ denotes the power spectrum of the shaped noise. Similarly, when receveiving 
two out of four descriptions (i.e. one of the pair of descriptions (0,2) or (1,3)) the side distortion d2 is 
given by 

d2 = 7^ / S,{oj)dLO + ^ [ S,{oj)dio (46) 

where the latter term is due to aliasing (since we downsample by two without applying any anti-aliasing 
filter). Finally, if we receive only a single description and thereby downsample by four, the side distortion 

"Notice that even fractional oversampling ratios can be used in which case we might also have aliasing of the source spectrum. 
'"We suspect that results from non-uniform sampling or non-uniform filterbank theory will prove advantageous for constructing 
the optimal decoders in the most general situation. However, this is a topic of future research. 
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di is given by the complete shaped noise spectrum, that is 

1 r 
di = — S^{uj)duj. 



(47) 



Once again, we let p ^ cx) and take advantage of the frequency-domain interpretation, which we 
previously presented for the case of two descriptions. We divide the power spectrum of the shaped noise 
into three flat regions as shown in Fig. [121 The low frequency band (i.e. \lo\ < 7r/4) is of power 60, the 
middle band (i.e. 7r/4 < \io\ < 37r/4) is of power 62, and the high band (i.e. 37r/4 < \io\ < vr) is of 
power 61. With this choice of noise shaping, we guarantee that c{z) is minimum phase simply by letting 
S2 = so that /^^ log2 Seiuj)duj = 0. From (|45]l - (07]) it follows tha^l 



and 



dc = ^So, 



d2 = ^{So + 5,] 
a? 



dc + -^Si 



di 



4 

d2 + 



(<5o + Si + 2/^/6o6l) 



(48) 



(49) 



(50) 



|c(e^-)|V|, 



So 



S2 = I/^/SqSi 



37r 
4 



Fig. 12. An example of a shaped noise power spectrum |c(e-'"^)| for K = 4 descriptions. 

The description rate follows easily from previous results since the source is memoryless after down- 
sampling. Specifically, it is easy to show that when using memoryless entropy coding the rate is given 



'^For clarity we have excluded the post multipliers, which are required for optimal reconstruction at general resolution. At 
high resolution conditions, the post multipliers become trivial and will not affect the distortions. 
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R = - lo" ' 



by 

°-j'*^V 4 

where the approximation becomes exact at high resolution. 

It is worth emphasizing that in this example we have two controlling parameters, i.e. 6q and 6i, where 
5o < 1 and 5q5i < 1. It is therefore possible to achieve almost arbitrary distortion ratios di/d2,di/dc 
and d^jdc- It was recently shown, see [39], that it is also possible to use several distortion controlling 
parameters in the source-splitting design of Chen et al. [15] and furthermore, by exploiting random 
binning, the achievable ii'-channel rate region of Pradhan et al. [7], [8] can be achieved. Random binning 
can also be used to enlargen the rate region of the index-assignment based schemes, cf. [13], [40]. 

For the case of distributed source coding problems, e.g. the Wyner-Ziv problem, efficient binning 
schemes based on nested lattice codes have been proposed by Zamir et al. [41]. However, these binning 
schemes are not (directly) applicable for the MD problemo An alternative binning approach based on 
generalized coset codes has recently been proposed by Pradhan and Ramchandran [42]. It was indicated 
in [42] that the coset-based binning approach is applicable also for MD coding but the inherent rate 
loss was not addressed. Thus, the problem of designing efficient capacity achieving binning codes for 
the MD problem appears to be unsolved. From a practical point of view, it is therefore desirable to 
avoid binning. While the proposed MD design based on Delta-Sigma quantization avoids binning, we 
do not know whether there is a price to be paid in terms of rate lossilj We Leave it as a topic of 
future research, to construct optimal reconstruction rules for the cases of non-uniform downsampling and 
furthermore addressing the issue whether the achievable ii'-channel rate-distortion region coincide with 
the one obtained by Pradhan et al. [7], [8]. 

VIII. Conclusions and Discussion 

We proposed a symmetric two-channel MD coding scheme based on dithered Delta-Sigma quantization. 
We showed that for large vector quantizer dimension and large noise shaping filter order it was possible 
to achieve the symmetric two-channel MD rate-distortion function for a memoryless Gaussian source and 

'"^By making use of time-sharing, it is possible to apply the binning schemes presented in [41] to the MD problem. 

'^Notice that we can reduce the coding rate by undersampling the signal so that the source spectrum will contain aliasing. As 
more descriptions are received, the lesser aliasing and a better reconstruction quality can be achieved. This stands in contrast to 
binning, where one can usually not reconstruct at all when too few descriptions are received. 
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MSE fidelity criterion. The construction was shown to be inherently symmetric in the description rate 
and there was therefore no need for source-splitting as were the case with existing related designs. It 
was shown that by simply increasing the oversampling ratio from two to K it was possible to construct 
K descriptions. The design of optimal reconstruction rules for > 2 descriptions was left as an open 
problem. Currently, we are working on extending the scheme to include prediction, in order to make it 
optimal for encoding sources with memory, without requiring entropy coders with memory, see [38]. 



We first find the central distortion (at high-resolution) through a time-domain approach and use the 
insights in order to find the optimal (3, which will then lead to the central distortion at general resolution. 

Let tn = Xn — Xn be the error signal. Without loss of generality, we may view the upsampling operation 
followed by ideal lowpass filtering as an over-complete expansion of the source, where the infinite- 



Appendix 



Proof of Lemma |4] 



dimensional analysis frame vectors with coefficients /i^ ^ 
Thus, adopting the notation of [25], we have that 



,n 



sinc(^^) are translated 





n=— oo 



and the synthesis filters are given by hk,' 



isinc(^^), so that 




Since Uk = ak + ek + Ya=i CiCk-i, the error e 



Xn — Xn is given by 




(51) 



The (per sample) mean squared error (MSE) is (by use of dST]) ) given by 





16i 



'The sine function is defined by 
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P 



1- 



oo oo 



X I ^CiEi 

oo 

E E 

fc=— oo (=— oo 

f p \ f ^ 

^ CiEk-i ^ CiE^ 



n — k\ f n — I 
sine I — - — sine 



i-i 



oo oo 



.2 = 



I E E 

k=—oo l=~oo 

X E[Efc-iS«-i 
-. oo 

I E 



n — k\ . ( n — l\ -J^ -J^ 
sine I — z — I sine ( — z — I > / ^ 



CjC 



i=0 j=0 



p p 



X E 



l=—oo 



X) 

E 



77- — / 

sine I 1 



(a) 1 \ ^ . I n — k \ \ ^ \ ^ 

A:=— oo 



X E 



El, ,sine 



^ j=0 j=0 

n — k — i + j 



9 p p 
(6) ct|; ^ ^ . ( i-J , 



(52) 



i=0 j=0 

where (a) follows from the faet that KE^^iEi^j is non-zero only when I — j = k — i whieh implies 
that I = k — i + j and (6) is due to the following property of the sine funetion [43] 



E. / ^\ ■ f k-ci 
sine ( Co ) sine ( cq 

k=—oo 



r smc I — 

r 



r J \ r 

Let U denote the reeonstruetion before the eentral post multiplier /?, i.e. U is the variable obtained 
by first lowpass filtering and then downsampling by two. It follows immediately that E?7^ = a\ + 
27 i|t^|<-=^ \c{e?^y\^ a\duj = a\ + a'^P^^. Furthermore, from (l52l) it ean be seen that 

2 P P 



i=0 j=0 



sine I ^—r^ I CiCj. 



(53) 



so using that E[X|C/] = PU yields 



0" 



X 



^1 + ^ ELo Ei=o sine( V)qcj 



'X 



(54) 
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The central distortion at general resolution now follows by inserting (3 (154] ) into (I24b . which leads to (I35l ). 

We will now derive the side distortion. First notice that since we only receive either all odd samples 
or all even samples, we should only sum over terms in (l52l ) where the lag |i — j| is even. However, all 
cross-terms, CiCj,i 7^ j, vanish since sinc(x/2) = for x = ±2, ±4, . . . , so only the p + I auto-terms, 
cf,i = 0, . . . ,p, contribute to the distortion. In addition, we make use of the following property of the 
sine function [43] 

E ^'^^ (v) ''''' (^) = ^^^"'^ © • (^^^ 

k=—oo 

With this, it follows that the high-resolution side distortion d^^ is given by 



j/ir ^2 „2 2 D 



i=0 

At this point, we let Ui denote the reconstruction before the multiplier a such that Xi = aUi,i = 0, 1. It 
should be clear that EXf/j = cr^i^ Recall that the autocorrelation of the even lags of Ui vanish so that 

V 

= al+al^cf = al+ alP^^ (56) 

Since, E[X|[/i] = aUi, it follows that 

- 4 

" - 2 , 2 y-P 3- (5'^) 

Inserting dSVl ) into ((22]) leads to (l36] ). which is the side distortion at general resolution. This completes 
the proof. ■ 

Proof of Lemma [5] 

A minimum-phase filter H{z) with power spectrum S{f) = \H{e^'^'"f)\'^, —1/2 < / < 1/2 satisfies 

where /iq is the zero-tap of the filter. It is also known that the zero-tap of a minimum-phase filter is 
strictly larger than the zero-tap of a non-minimum-phase filter having the same power spectrum [44]. 
Thus, for an arbitrary filter H{z) with power spectrum S{f) and zero-tap ho 



e 



|2 



"The even samples are noisy versions of X where the noise is independent of X. The odd samples are noisy and phase 
shifted versions of X. However, the phase shift is corrected by the all-pass filter hp{z) before the post multiplier. Thus, 
KXU, = o\,i = Q,l. 
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with equality if and only if H{z) is minimum phase. Furthermore, from the geometric-arithmetic means 
inequality it can be shown that 




S{f)df 5(/)d/>e^-v.i-^(/)'^/ (58) 

|/|<l/4 Jl/4<|/|<l/2 

> 1 (59) 



where we used the fact that in our case the filter is monic, so /iq = 1 and where we have equality in (1581 ) 
and (|59l ) if and only if the filter H{z) is minimum phase and the power spectrum consists of two fiat 
regions; S{f) = for |/| < 1/4 and S{f) = 6 for 1/4 < |/| < 1/2. Let us now fix the energy ratio 
PdjPd. = 1, where 1 < 7 e M, P,, = f^^^^,^, Sif)df and P,^ = Jj^|<,/, 5(/)d/ + /i/4<|/|<i/2 S{f)df. 
With this, it follows that 

/l/4<|/|<l/2^W/^^_^^ (60) 
ii/|<l/4^(/)^/ 

Using (l60l) in the left-hand-side of (1581 ) leads to the following two inequalities 
and 

P..>^^/^+^^ = ^(<^ + r^) (62) 

where we have equality in both (|6TI ) and (l62l ) (at the same time) if and only if the filter is minimum phase 
and the power spectrum is a two-step function, i.e. it has constant power = 1 / 1/7 — 1 through-out 
the lowpass band and 5 through-out the highpass band. 

2 2 

At this point we let a = -2 n and (3 = 2 n from which it can be shown that the distortions 
at general resolution are given by 

= — 2~5~ (63) 

+ Vd. 



ds = 2\t2:. ■ (64) 



and 

cy\ + alPd 

Inserting the lower bounds of ( [6II ) and ( [62l ) into ( [63] ) and ( [64] ) yields Ozarow's symmetric rate-distortion 
function (see (|25] ) and (|23])). Moreover, (|63] ) and ( [64l) are strictly increasing in P^^ and P^^, respectively. 
Thus, for a fixed ratio 7, any other spectrum than the two-step S'(/) given above must necessarily lead 
to a greater distortion. To complete the proof, we remark that in order to have such an ideal brick-wall 
power spectrum, the order of the filter must necessarily be infinite. ■ 
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